A Fast and Convergent Stochastic Learning Algorithm for MLP

We propose a stochastic learning algorithm for multilayer perceptrons of linearthreshold function units, which theoretically converges with probability one and experimentally (for the three-layer network case) exhibits 100% convergence rate and remarkable speed on parity and simulated problems. On the parity problems (to realize the n bit parity function by n (minimal) hidden units) the algorithm converged on all the trials we have tested (n = 2 to 12) with average of 5.2 · 4.2n presentations where n is the number of bits and spent about 26 minutes for n = 12 on average on a 533MHz Alpha 21164A chip. We do not need to adjust parameters for the whole experiments, which is not the case for error backpropagation algorithms.